Effectiveness of web search results for genre and sentiment classification

The motivation of this study is to enhance general topical search with a sentiment-based one where the search results (called snippets) returned by the Web search engine are clustered by sentiment categories. Firstly we developed an automatic method to identify product review documents using the snippets (summary information that includes the URL, title, and summary text), which is considered as genre classification. Then the identified snippets of product review documents were automatically classified into positive (recommended) and negative (non-recommended) documents, which is sentiment classification. Thereafter the user may directly decide to access the positive or negative review documents. In this study we used only the snippets rather than their original full-text documents, and applied a common machine learning technique, SVM (Support Vector Machine), and heuristic approaches to investigate how effectively the snippets can be used for genre and sentiment classification. For genre classification, the hybrid approach which made use of both the machine learning approach using n-gram terms and a heuristic approach using the Title, Summary Text and the URL performed slightly better than the machine learning approach alone. The best result was 84.79% accuracy with unseen 400 snippets. For sentiment classification, when we used only the snippets from PC Magazine review site and their summary texts were only used as a document feature to see whether or not the search engine provides good quality summaries (i.e. summary text) sufficient for sentiment classification, the classification accuracy was rather low. The best result was 70.73% accuracy with unseen 164 snippets. The experiment results and error analyses show that the Web search engine should improve the quality of the snippets especially for opinionated documents (i.e. review documents).